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Abstract 


The  research  presented  in  this  thesis  focuses  on  developing,  demonstrating,  and 
evaluating  the  concept  of  a  Cursor-on-Target  control  system  for  semi-autonomous 
unmanned  aircraft  systems.  The  Department  of  Defense  has  mapped  out  a  strategy  in 
which  unmanned  aircraft  systems  will  increasingly  replace  piloted  aircraft.  During  most 
phases  of  flight  autonomous  unmanned  aircraft  control  reduces  operator  workload, 
however,  real-time  information  exchange  often  requires  an  operator  to  relay  decision 
changes  to  the  unmanned  aircraft.  The  goal  of  this  research  is  to  develop  a  preliminary 
Cursor-on-Target  control  system  to  enable  the  operator  to  guide  the  unmanned  aircraft 
with  minimal  workload  during  high  task  phases  of  flight  and  then  evaluate  the  operator’s 
ability  to  conduct  the  mission  using  that  control  system.  For  this  research,  the  problem  of 
Cursor-on-Target  control  design  has  multiple  components.  Initially,  a  Cursor-on-Target 
controller  is  developed  in  Simulink.  Then,  this  controller  is  integrated  into  the  Aviator 
Visual  Design  Simulator  to  develop  an  operator-in-the-loop  test  platform.  Finally,  a 
ground  target  is  simulated  and  tracked  to  validate  the  Cursor-on-Target  controller.  The 
Cursor-on-Target  control  system  is  then  evaluated  using  a  proposed  operator  rating  scale. 
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To  my  best  friend  in  life 


“The  nation  that  draws  too  great  a  distance  between  its  soldiers  and  its  scholars  will  have 
its  children  taught  by  cowards  and  its  fighting  done  by  fools.” 

Thucydidus,  400  B.C. 

“It  is  not  the  critic  who  counts,  not  the  man  who  points  out  how  the  strong  man  stumbled, 
or  where  the  doer  of  deeds  could  have  done  better.  The  credit  belongs  to  the  man  who  is 
actually  in  the  arena;  whose  face  is  marred  by  the  dust  and  sweat  and  blood;  who  strives 
valiantly;  who  errs  and  comes  short  again  and  again;  who  knows  the  great  enthusiasms, 
the  great  devotions  and  spends  himself  in  a  worthy  cause;  who  at  the  best,  knows  in  the 
end  the  triumph  of  high  achievement,  and  who,  at  worst,  if  he  fails,  at  least  fails  while 
daring  greatly;  so  that  his  place  shall  never  be  with  those  cold  and  timid  souls  who  know 
neither  victory  or  defeat.” 


President  Theodore  Roosevelt,  A.D.  1910 
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DEVELOPMENT  OF  CURSOR-ON-TARGET  CONTROL 
FOR  SEMI- AUTONOMOUS  UNMANNED  AIRCRAFT  SYSTEMS 


I.  Introduction 


1.1  Overview 

The  method  to  control  Unmanned  Aircraft  (UA)  varies  greatly  from  remotely 
piloted  to  fully  autonomous.  Remotely  piloted  control  requires  full  human  involvement 
with  the  operator  controlling  all  actions.  Fully  autonomous  control  does  not  require  any 
human  involvement  and  the  flight  path  typically  relies  on  waypoints  for  the  vehicle  to  fly 
along.  A  necessary  blend  in  many  applications  is  semi-autonomous  control,  in  which  the 
ground  operator  can  decide  at  which  point  to  enter  the  control  loop  and  only  gives  high 
level  commands.  The  ground  operator  closes  the  control  loop  by  observing  vehicle  flight 
parameters  (Breneman,  1992). 

The  majority  of  military  UA  carry  optical  sensor  equipment  and  relay  images 
from  those  sensors  back  to  a  ground  station.  These  images  provide  a  picture  of  the 
operational  environment  and  enable  the  operator  to  make  decisions.  This  source  of 
information  is  a  passive  source  of  infonnation,  and  would  not  readily  alert  targets  that 
they  are  being  tracked.  In  missions  that  involve  tracking  moving  targets,  such  as  tanks  on 
land  or  ships  at  sea,  visual  tracking  is  practical  because  continual  geographical 
coordinates  are  not  required  for  the  target. 
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1.2  Cursor-on-Target  Definition 


The  concept  of  Cursor-on-Target  (COT)  describes  an  alternative  way  for  an 
operator  to  control  a  semi-autonomous  UA.  The  Cursor-on-Target  controller  would 
allow  the  operator  to  place  a  cursor  over  the  target  of  interest  on  data  linked  video  from 
the  UA  and  therefore  direct  the  UA  to  track  the  target  while  at  the  same  time  collecting 
information.  Depending  on  where  the  cursor  is  placed,  the  UA  would  steer  toward  the 
target  and  adjust  its  speed  to  direct  the  target  to  the  middle  of  the  display. 

1.3  Alternative  Department  of  Defense  Definition  of  Cursor-on-Target 

The  Department  of  Defense  (DoD)  has  previously  defined  the  term  “Cursor  on 
Target  (CoT)”  and  therefore  their  definition  will  briefly  be  explained  to  avoid  confusion 
from  the  use  of  the  definition  in  this  research  paper.  In  April  2002,  Air  Force  Chief  of 
Staff  General  Jumper  stated  his  vision  that  “the  sum  of  all  wisdom  is  a  cursor  over  the 
target.”  This  led  to  a  software  development  by  federally  funded  MITRE  Corporation  for 
CoT  (Byrne,  2005).  CoT  allows  different  communities  in  the  various  Services  the  ability 
to  share  vital  information  near  real-time.  CoT  leverages  the  widespread  XML 
(extensible  Markup  Language)  and  defines  a  common  extensible  message  format  for 
communicating  key  targeting  infonnation  (what,  when,  where).  The  goal  is  for  an 
operator  to  reference  the  operational  picture  display,  put  his  cursor  over  the  target  in 
question  and  upon  clicking  on  the  target  receive  tasking  orders  and  data  about  the  target 
from  multiple  sources.  The  U.S.  Joint  Forces  Command  conducted  an  experiment 
incorporating  CoT  with  UAS.  CoT  allowed  commanders  in  the  field  to  obtain 
information  from  various  UA  and  even  allowed  remote  control  of  sensor  video 
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(Roosevelt,  2005).  Without  the  program  details  being  published,  it  is  hard  to  determine 
the  level  that  MITRE’s  CoT  technology  will  enable  control  of  UA  (Miller,  2004). 

1.4  Problem  Statement 

Although  autonomous  UA  control  during  most  phases  of  flight  reduces  operator 
workload,  real-time  information  exchange  often  requires  an  operator  to  relay  decision 
changes  to  the  UA.  The  goal  of  this  research  is  to  develop  and  evaluate  a  Cursor-on- 
Target  control  system  that  enables  an  operator  to  track  moving  targets  with  minimal 
workload  even  during  high  task  phases  of  flight. 

1.5  Proposed  Solution 

This  research  will  develop  a  control  interface  that  allows  the  operator  to  control  a 
UA  by  simply  positioning  a  cursor  on  display.  The  controller  which  connects  the  camera 
video  to  the  aircraft  equations  of  motion  will  first  be  modeled  in  Simulink  (The 
MathWorks,  2004).  This  combination  will  then  be  integrated  into  the  Aviator  Visual 
Design  Simulator  (AVDS)  (Rasmussen,  2005).  When  desired,  the  operator  will  then  take 
over  Cursor-on-Target  control  and  guide  the  aircraft. 

1.6  Thesis  Objectives 

The  objective  of  the  proposed  research  is  three-fold.  The  first  objective  is  to 
develop  a  COT  controller.  This  objective  will  enable  an  operator  without  military  pilot 
training  the  ability  to  control  a  UA.  The  second  objective  is  to  demonstrate  the  feasibility 
of  COT  control  by  testing  various  real-world  scenarios.  This  objective  will  be  tested  by 
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taking  untrained  operators  and  allowing  them  to  try  out  the  developed  controller  in  an 
opera tor-in-the-loop  simulation.  The  third  objective  will  be  to  evaluate  the  COT 
controller  through  measures  of  performance  based  upon  a  Cooper-Harper  type  rating 
scale. 

1.7  Background  on  Department  of  Defense  UAS 

1.7.1  Roadmap  for  Department  of  Defense  Unmanned  Aircraft  Systems 

The  Department  of  Defense’s  “Unmanned  Aircraft  Systems  Roadmap,  2005- 
2030”  is  a  foundational  document  for  any  discussion  concerning  United  States  military 
UA.  In  the  roadmap,  the  stated  goal  is  “to  guide  the  Department  toward  a  logical, 
systematic  migration  of  Unmanned  Aircraft  Systems  (UAS)  mission  capabilities  focused 
on  the  most  urgent  warfighter  needs  (United  States,  2005).”  The  twenty-five  year  span 
allows  identification  of  future  timeframes  when  technology  should  become  available  and 
covers  a  typical  generation  of  aircraft  from  laboratory  research  to  fielded  system. 

Although  intended  to  be  strong  guidance,  each  Service  is  ultimately  responsible 
for  prioritizing  requirements  and  authorizing  specific  UAS.  In  March  2007,  Air  Force 
Chief  of  Staff  General  Moseley  proposed  that  the  Air  Force  should  gain  oversight  for 
developing  UA  that  operate  above  3,500  feet  in  order  to  avoid  duplication  of  service 
acquisition  efforts,  decrease  cost  in  developing  new  technologies,  and  standardize 
operations  and  training  (Rolfsen,  2007;  Allard,  2007).  A  recent  House  Armed  Services 
subcommittee  hearing  on  the  proposal  received  opposition  from  Army,  Navy,  and  Marine 
leadership.  This  opposition  was  due  to  an  anticipated  lack  of  connection  by  the  Air  Force 
with  specific  Service  needs  should  they  become  the  sole  Service  responsible  for  higher 
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altitude  UA.  This  discussion  is  not  finalized  and  the  separate  services  are  now  required 
to  submit  in  writing  why  they  disagree  with  the  Air  Force  proposal  (Wehrman,  2007). 

1.7.2  Funding  for  Department  of  Defense  Unmanned  Aircraft  Systems 

The  clearest  demonstration  of  DoD  support  for  UAS  can  be  seen  by  observing 
funding  profiles.  Between  1990  and  1999,  over  $3  billion  was  invested  in  UAS 
development,  procurement,  and  operations.  This  amount  grew  to  over  $4  billion  between 
2000  and  2004,  with  fiscal  year  2003  being  the  first  billion-dollar  year  in  history. 

Between  2005  and  2006  another  $4  billion  was  spent,  with  fiscal  year  2005  being  the  first 
$2  billion-year.  The  next  four  years,  2007-2011,  will  again  see  increased  spending  to 
exceed  $13  billion  (United  States,  2005).  Such  funding  profiles  show  resolve  to  see  the 
roadmap  goals  become  a  reality. 

1.7.3  Why  Unmanned  Aircraft  Systems? 

Since  so  many  resources  are  being  dedicated  toward  UAS  development,  it  is  wise 
to  ask  what  advantage  a  UA  has  over  manned  aircraft.  The  latest  Roadmap  focuses  on 
UAS  in  order  to  emphasize  that  UA  are  only  one  component  of  a  system  that  includes  the 
operator,  ground  station,  and  satellites  if  necessary.  A  UA,  by  definition  is  “a  powered, 
aerial  vehicle  that  does  not  carry  a  human  operator,  uses  aerodynamic  forces  to  provide 
vehicle  lift,  can  fly  autonomously  or  be  piloted  remotely,  can  be  expendable  or 
recoverable,  and  can  carry  a  lethal  or  non-lethal  payload.  Ballistic  or  semi-ballistic 
vehicles,  cruise  missiles,  and  artillery  projectiles  are  not  considered  unmanned  aerial 
vehicles  (United  States,  2005).”  The  catch-phrase  “dull,  dirty,  or  dangerous”  (DDD) 
describes  those  missions  in  which  UA  make  a  strong  case  for  replacing  manned  aircraft. 
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Dull  missions  such  as  long-range  bombing  or  Intelligence,  Surveillance  and 
Reconnaissance  (ISR)  flights  require  long  periods  of  alertness.  A  rotating  crew  on  the 
ground  can  match  the  sustained  in-flight  time  of  a  UA.  For  dirty  missions  such  as  flights 
in  nuclear,  chemical,  biological,  and  radiological  environments,  as  well  as  dangerous 
missions  such  as  reconnaissance  and  suppression  of  enemy  air  defenses  (SEAD)  flights, 
the  removal  of  human  causalities  if  the  mission  is  lost,  and  higher  confidence  in  mission 
success  are  strong  motivators  for  UA  (United  States,  2005). 

1.7.4  United  States  Navy  Broad  Area  Maritime  Surveillance  (BAMS) 

Concept 

Since  all  manned  flights  can  in  some  way  be  categorized  as  DDD,  one  must 
understand  that  the  Roadmap  calls  for  a  gradual  transition  from  manned  to  unmanned 
flight  in  areas  that  overlap  capabilities  or  require  new  technology.  A  key  question  to  be 
asked  prior  to  UAS  development  is  what  requirements  for  military  capabilities  could 
potentially  be  filled  by  UAS.  This  requires  forethought  and  preparation  rather  than 
creating  a  UA  for  every  situation.  The  Navy  has  answered  this  question  with  a  concept 
termed  Broad  Area  Maritime  Surveillance  (BAMS)  (see  Figure  1). 


Figure  1.  Navy  BAMS  UAS  Program  (U.S.  Navy) 
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BAMS  is  the  Navy’s  unique  venture  to  blend  the  capabilities  of  UA  and  manned 
aircraft.  The  concept  resulted  from  the  immediate  need  to  replace  the  entire  P-3C  Orion 
fleet.  The  P-3C  aircraft  is  the  Navy’s  workhorse  to  conduct  anti-submarine  warfare 
(ASW),  anti-surface  warfare,  and  overland  ISR  missions  (see  Figure  2).  The  P-3C  is  an 
aging  airframe  and  requires  a  replacement  by  2013  as  the  fleet  has  already  dwindled  from 
greater  than  300  aircraft  to  approximately  150  aircraft.  Many  of  the  P-3C  missions, 
especially  ASW,  require  a  manned  aircrew.  Regardless  of  this  fact,  healthy  debate  over 
UAS’s  role  in  the  future  occurred  when  the  Navy  conducted  an  Analysis  of  Alternatives 
in  2000.  The  analysis  “identified  manned  aircraft  as  an  essential  element  of  the  suite  of 
systems  that  will  satisfy”  the  needed  missions  for  the  next  generation  of  aircraft  (United 
States,  2003).  Thus,  a  new  Multi-Mission  Aircraft  concept  was  created  and  is  now 
identified  as  the  P-8A,  a  militarized  Boeing  737-800  (see  Figure  2). 


Figure  2.  The  P-3C  and  P-8A  ;  The  past,  present,  and  future  of  Maritime  Patrol  (U.S.  Navy) 
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1.7.4.1  Details  of  BAMS  Unmanned  Aircraft  Systems  and  the  P-8A 


BAMS  UAS  will  operate  in  conjunction  with  the  P-8A.  Starting  in  2013,  108 
P-8  A  aircraft  along  with  110  BAMS  UAS  will  gradually  replace  the  currently  operating 
P-3C  Orion  (United  States,  2006).  All  current  missions  will  continue  to  be  perfonned  by 
the  manned  P-8A  platform.  BAMS  UAS  will  primarily  be  developed  to  provide 
persistent,  worldwide  maritime  ISR  capability.  The  UAS  will  be  unarmed,  possess  high 
endurance,  and  will  operate  from  land-based  sites  worldwide  (see  Figure  3).  Eventually 
each  BAMS  UAS  site  will  support  five  to  six  vehicles  that  will  be  capable  of  24  hour,  7 
days  a  week  coverage  to  on-station  ranges  of  2,000  nautical  miles  (Pike,  2006).  In 
addition  to  contributing  a  reliable  U.S.  operational  picture  of  the  battlespace,  BAMS 
UAS  will  provide  communication  relay  capability  in  a  “low  hanging  satellite”  role 
(United  States  2005).  Many  options  are  currently  being  considered  as  the  BAMS  UAS 
and  the  final  decision  is  expected  in  July  2007. 


Figure  3.  BAMS  UAS  -  Global  Hawk  variant  is  one  possibility  (U.S.  Air  Force) 
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1. 7.4.2  BAMS  Micro-UA  Proposal 


In  addition  to  BAMS  UAS,  Sonochute-Launched  expendable  UA  (S-LUA)  are 
also  being  developed  to  augment  the  P-8A.  The  purpose  of  such  S-LUA  is  twofold;  to 
provide  standoff  ISR  capability,  and  reduce  engine  wear  and  fuel  burned.  One  prototype 
is  the  Coyote  designed  by  Advanced  Ceramics  Research  in  2004  (see  Figure  4).  The 
Coyote  had  a  Level  IV  demonstration  in  July  2006,  which  enabled  S-LUA  and  sensor 
payload  command  and  control  (see  Table  1).  At  a  unit  cost  of  less  than  $10,000,  it  could 
realistically  cost  less  than  a  P-3C  or  P-8A  descending  from  altitude,  cruising  at  sea  level, 
and  then  returning  to  altitude.  It  will  be  launched  from  sonobuoy  tubes,  weighs  14  lbs, 
has  a  range  of  20  nm  and  endurance  of  1 .5  hours  on  electric  power  (Peck,  2006).  The 
Coyote  is  programmable  while  still  in  the  launch  mode,  and  at  release,  goes  into  a  glide 
mode  for  a  fast  descent  at  100  knots.  Once  on  station,  it  transitions  to  a  cruise  speed  of 
approximately  75  knots  and  follows  GPS-based  waypoints  in  automatic  mode.  It  can  also 
be  controlled  by  a  line-of-sight  datalink.  The  Coyote  can  carry  either  electro-optical 
(EO)  or  infrared  (IR)  cameras  (Sherman,  2006).  Initial  production  included  a  fixed 
camera  and  research  is  being  conducted  to  explore  an  inexpensive  gimbaled  camera 
(Goodall,  2005). 


Figure  4.  Coyote  Sonochute-Launched  expendable  UA  (S-LUA)  (Advanced  Ceramics  Research) 
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1.7.4.3  Funding  for  BAMS  and  the  P-8A 


Current  budgetary  constraints  do  not  allow  for  enough  P-8A  aircraft  to  be 
procured  in  order  to  completely  replace  the  P-3C  fleet  and  fulfill  all  needed  missions. 
Each  of  the  projected  108  P-8As,  based  on  Boeing’s  737-800,  cost  $172  million  for  a 
total  cost  of  $18.5  billion.  This  financial  restriction  became  a  golden  opportunity  for  the 
Navy  to  adapt  UAS  to  cover  many  of  the  DDD  roles  that  the  P-3C  had  performed.  The 
BAMS  UAS  has  been  given  a  budget  of  over  $1.3  billion  from  FY05-FY 1 1  with  the  type 
of  platform  to  be  decided  in  the  summer  of  2007  (United  States,  2005).  The  Coyote  is 
another  relatively  inexpensive  development  with  less  than  $800,000  devoted  to  testing  as 
of  January  2006  (Peck,  2006).  Once  the  programs  combine  in  the  operational  world,  the 
unparalleled  worldwide  capabilities  provided  will  prove  a  valuable  return  on  investment. 

1. 7.4.4  P-8A  Command  and  Control  of  BAMS 

Once  on-station,  the  P-8A  will  be  able  to  correlate  data  from  BAMS  UAS  in  order 
to  achieve  infonnation  superiority.  The  initial  production  P-8  A  will  be  capable  of  Level 
II  Command  and  Control  (see  Table  1),  which  will  allow  near  real-time  receipt  of  UA 
sensor  data  via  direct  link.  Later  production  blocks  of  P-8  A  will  be  capable  of  Level  IV 
Command  and  Control,  which  will  enable  UA  and  sensor  payload  command  and  control 
(United  States,  2006).  The  ability  to  connect  the  P-8A  with  BAMS  UA  or  S-LUA  will 
greatly  magnify  the  reach  of  fleet  commanders  as  demonstrated  in  the  following  scenario. 
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Table  1.  UA  Operations  Standards  (United  States,  2005) 


Level  1 

Indirect  receipt/transmission  of  UA  related  payload  data 

Level  2 

Direct  receipt  of  ISR/other  data  where  “direct”  covers  reception  of  the 

UA  payload  data  by  the  Unmanned  Control  Station  when  it  has  direct 
communication  with  the  UA 

Level  3 

Control  and  monitoring  of  the  UA  payload  in  addition  to  direct  receipt  of 
ISR/other  data 

Level  4 

Control  and  monitoring  of  the  UA,  less  launch  and  recovery 

Level  5 

Control  and  monitoring  of  the  UA  (Level  4),  plus  launch  and  recovery 
functions 

1.8  Potential  Real  World  Scenario 

Based  on  recent  intelligence  and  public  sources  (Halloran,  2007),  the  following 
motivational  scenario  is  a  real  possibility  and  one  that  clearly  shows  the  role  that  UA  and 
S-LUA  could  play  in  increasing  the  P-8A’s  capabilities. 

September  2015: 

The  People’s  Republic  of  China  (PRC)  has  launched  a  massive  military  exercise 
in  the  Taiwan  Straits  in  a  deliberate  attempt  to  put  intense  political  pressure  on  the 
Republic  of  China  (R.O.C.)  (see  Figure  5).  PRC  has  put  to  sea  fifty-five  naval  vessels,  of 
which  ten  PRC  diesel  submarines  are  unlocated.  This  naval  task  force  has  cut  off  all 
commercial  traffic  in  the  Straits  and  to  Taiwan.  In  addition,  PRC  has  launched  multiple 
missile  tests  in  the  area,  with  one  missile  flying  over  the  island  of  Taiwan.  These  actions 
have  threatened  the  stability  of  the  entire  region,  and  intense  pressure  is  being  felt  by  the 
United  States  to  intervene  on  R.O.C’s  behalf. 


11 


Figure  5.  Map  of  Taiwan  Strait;  Assets  available  for  U.S.  response  to  the  potential  scenario 


Due  to  political  constraints,  no  U.S.  naval  surface  vessels  are  sent  into  the  volatile 
Straits.  However,  both  P-8A  and  BAMS  UAS  assets  are  stationed  in  Japan,  and  provide 
24  hour,  7  days  a  week  coverage  as  the  events  unfold.  The  P-8A  aircraft  provides  an 
annament  platform  to  demonstrate  the  strength  of  the  U.S.  but  remains  outside  of  the 
weapon  ranges  of  PRC  surface  vessels.  The  P-8A  platform  can  task  either  BAMS  UA  or 
S-LUA  to  fly  into  the  Straits  and  maintain  ISR  coverage  of  the  unfolding  events.  In  the 
event  the  UA  is  lost,  there  would  be  no  loss  to  human  life,  while  at  the  same  time  the 
U.S.  can  claim  a  direct  attack  against  sovereign  U.S.  property. 

To  complicate  matters,  all  satellite  coverage  and  communications  have  been  lost 
for  unknown  reasons,  including  Global  Positioning  Satellites.  The  BAMS  UA  is  able  to 
proceed  on-station  via  INS.  Once  on-station,  Command  and  Control  by  P-8A  is 
necessary  to  acquire  and  track  the  constantly  moving  naval  vessels.  An  operator  onboard 
P-8A  is  able  to  direct  either  BAMS  UA  or  S-LUA  below  an  8,000  to  12,000  foot  cloud 
deck  to  allow  EO  and  IR  sensors  to  acquire  the  needed  type  and  name  of  each  individual 
surface  vessel  by  COT  control.  This  information  provides  a  complete  surface  picture  of 
PRC  capabilities  to  fleet  commanders. 


12 


1.9  Thesis  Outline 


In  this  Chapter,  an  introduction  of  the  concept  of  COT  was  given,  to  include  the 
DoD’s  and  specifically  the  Navy’s  future  roadmap  for  UA.  Chapter  II  reviews  published 
research  applicable  to  COT,  specifically  focusing  on  interface  options  for  UAS  control 
and  vision  based  navigation.  Chapter  III  discusses  the  assumptions  for  COT  design, 
sensor  information,  electro-optic  target  acquisition  process,  and  vehicle  equations  of 
motion.  Chapter  IV  describes  in  detail  the  development  of  COT  and  provides  diagrams 
of  the  simulation  for  visual  reference.  Chapter  V  proposes  a  UAS  operator  rating  scale. 
Chapter  VI  lays  out  a  test  plan  with  scenarios,  describes  the  measures  of  performance 
used  to  evaluate  the  COT  performance,  and  discusses  the  test  results.  Chapter  VII 
summarizes  completed  research  and  lists  recommendations  for  improvement  for  follow- 
on  research. 
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II.  Literature  Review 


2.1  Interface  Options  for  UAS  Control 

Human  operators  will  always  remain  in  the  control  loop  of  UA  control  to  some 
degree  due  to  the  dynamic  nature  of  flight  and  the  situations  in  which  UA  will  be  used. 
However,  the  way  humans  will  interface  with  UA  may  change  dramatically  over  the 
years.  The  Roadmap  states  that  for  “those  UA  remaining  under  human  control,  the 
controller  will  eventually  be  linked  to  his  remote  charge  through  his  own  neuromuscular 
system  (United  States,  2005).”  Electrical  signals  sent  to  muscles  will  translate  into 
instantaneous  control  inputs  to  UA.  Until  the  day  technology  advances  to  that  point,  the 
available  technologies  of  touch  control  screens,  voice  control,  cursor  control,  and  helmet 
mounted  displays  provide  a  wide  variety  of  options  for  interfacing  with  UAS.  A  review 
of  research  conducted  on  these  options  provides  a  good  point  of  reference  for  why  COT 
is  a  viable  option  for  the  proposed  research. 

2.1.1  Target  Designation  by  Speech,  Joystick,  and  Touch  Control:  Format 

Design  Implications  (Curry,  1985) 

In  an  investigation  at  Wright-Patterson  Air  Force  Base  in  1985,  the  Flight 
Dynamics  Faboratory  compared  target  designation  by  standard  joysticks,  touch  sensitive 
overlays,  and  an  unconstrained  voice  recognition  system  versus  the  traditional  keying  in 
of  the  geographical  coordinates  and  elevation.  In  this  experiment,  pilots  wore  flight  gear 
and  equipment  for  use  in  a  chemical,  biological,  and  radiological  environment.  The  study 
was  clear  in  pointing  out  the  advantages  and  disadvantages  of  the  different  methods.  The 
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results  showed  that  the  touch-sensitive  control  mode  was  significantly  faster  than  that  of 
either  the  joystick  or  the  voice  recognition  mode,  while  there  was  no  significant 
difference  between  the  latter  two  modes.  Of  subjective  note,  the  joystick  was  found  to  be 
mentally  difficult  to  operate  due  to  the  inability  of  subjects  to  manipulate  the  cursor  using 
peripheral  vision.  The  opinion  of  which  mode  should  be  implemented  showed  an  even 
split  between  voice  and  touch  control.  However,  all  three  modes  were  superior  to  that  of 
keying  in  particular  geographic  coordinates  for  each  point. 

2.1.2  An  Examination  of  Cursor  Control  Techniques  to  Designate  Targets  on 

a  Cockpit  Map  Display  (Liggett,  1995) 

In  another  investigation  conducted  at  Wright-Patterson  Air  Force  Base  Advanced 
Cockpit  Branch  in  1995,  a  touch  control  system,  voice  control  system  and  traditional 
target  designation  control  (TDC)  on  the  throttle  were  compared.  Pilots  tested  the 
performance  of  these  controls  in  the  designation  of  targets  on  displays.  Results 
concluded  that  the  touch  control  system  always  provided  significantly  faster  designation 
times  when  compared  to  the  TDC,  and  the  TDC  was  significantly  faster  than  the  voice 
control.  The  intuitive  nature  of  the  touch  control  system  contributed  to  this  result.  In 
particular,  the  touch  control  system  was  recommended  for  multiple  moving  targets  on  a 
two-dimensional,  look-down  view  because  it  can  be  used  quickly  and  accurately. 
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2.1.3  Helmet  Mounted  Displays  for  Unmanned  Aerial  Vehicle  Control, 

(Morphew,  2004) 

In  a  2004  experiment  at  the  Ames  Research  Center,  Moffett  Field,  CA,  the  effect 
of  a  UA  operator  wearing  a  Helmet  Mounted  Display  (HMD)  versus  using  a  conventional 
computer  joystick  to  perform  a  sensor  target  search  task  was  assessed.  The  experiment 
explored  the  costs  and  benefits  associated  with  using  HMDs  for  UA  Ground  Control 
Station  operation.  The  objective  performance  measures  included  target  detection 
accuracy  and  response.  In  addition,  subjective  measures  included  fatigue,  situational 
awareness,  and  sickness.  The  operators  used  both  HMD  and  a  joystick  to  direct  the  UA’s 
sensor  to  search  for  targets  in  a  virtual  world  while  the  UA  flew  70  knots  at  an  altitude  of 
5,000  feet.  The  UA’s  sensor  had  a  six  degree  field  of  view  (FOV)  while  the  HMD  had  a 
horizontal  FOV  of  fifty  degrees.  This  resulted  in  a  magnification  effect  of  approximately 
seven  times.  The  sensor’s  slew  rate  of  sixty  degrees  per  second  required  a  limiter  to  be 
installed  on  the  HMD  to  better  approximate  the  actual  sensor  behavior.  The  results 
showed  no  significant  difference  in  operator  ability  to  correctly  classify  targets. 

However,  the  conventional  joystick  cursor  method  resulted  in  better  tracking  accuracy 
and  classification  farther  away  from  the  target.  On  a  negative  note,  the  HMD  resulted  in 
significant  simulator  sickness  including  nausea,  eye  strain,  and  disorientation.  The  study 
concluded  by  stating  that  advanced  technologies  like  HMD  should  only  be  implemented 
if  they  “demonstrate  an  improvement  over  the  existing  technology  in  the  desired 
dimension  (Morphew,  2004).” 
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2.1.4  Evaluation  of  a  Touch  Screen-Based  Operator  Control  Interface  for 
Training  and  Remote  Operation  of  a  Simulated  Micro-Uninhabited  Aerial 
Vehicle  (Cook,  2006) 

In  2004,  the  U.S.  Army  Research  Institute  evaluated  a  touch  screen-based 
operator  control  for  a  backpackable  micro  air  vehicle  (MAV).  The  operator  control  unit 
developed  by  Northrop-Grumman  took  a  unique  approach  to  teleoperation.  It  allowed 
manual  control  by  touching  a  spot  on  the  sensor  image  itself  and  directed  the  vehicle 
relative  to  what  is  seen  in  the  sensor  image.  The  MAV  was  equipped  with  a  forward- 
looking  and  downward-looking  fixed  camera  and  only  one  view  was  seen  at  a  given  time. 
Participants  rated  the  operation  as  difficult,  due  to  maneuvering  based  on  a  fixed  camera 
and  the  complex  rules  that  governed  the  translation  of  a  single  input  into  the  MAV 
response.  A  human  moving  through  space  has  several  sources  of  information,  and  can 
make  anticipatory  movements  toward  the  position  they  are  approaching.  In  the  video 
feed,  this  infonnation  is  largely  absent.  The  MAV  tested  was  a  vertical  takeoff  and 
landing  vehicle.  Touching  the  sensor  image  moved  the  MAV  forward  into  the  image. 
Each  touch  on  the  left  or  right  side  the  sensor  image  rotated  the  MAV  heading  by  an 
angle  proportional  to  the  distance  from  center.  Touching  below  the  center  reduced  the 
altitude  and  touching  above  the  center  maintained  altitude.  The  distance  and  speed  that 
the  MAV  moved  was  a  function  of  its  current  altitude.  In  summary,  one  input  had  the 
capability  of  moving  the  MAV  in  three  dimensions.  This  was  too  demanding  for  novice 
users,  and  the  study  concluded  that  it  would  be  “better  to  translate  a  single  touch  to 
up/down  and  left/right  (Cook,  2006).” 
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2.1.5  Summary  of  Interface  Applications  to  Cursor-on-Target 


Although  technology  is  improving,  cursors  and  more  advanced  touch  screens 
have  been  a  reliable  interface  choice  for  the  past  twenty  years.  This  is  especially  true  for 
multiple  moving  targets  as  noted  by  the  Advanced  Cockpit  Branch.  The  U.S.  Army 
Institute  article  is  of  particular  interest  because  of  the  close  similarity  to  the  current 
research  problem  (Cook,  2006).  The  goal  in  that  research  was  to  enable  an  untrained 
aircraft  operator  to  be  able  to  control  the  UA  with  a  touchscreen  interface.  The  main 
difference  between  that  research  and  the  current  problem  was  the  cursor  gave  updated 
position  coordinates  for  the  UA  to  fly;  the  current  research  has  the  cursor  actually 
controlling  the  flight  path  constantly.  The  main  lesson  to  be  learned  from  the  Army 
research  is  to  make  the  interface  and  cursor  inputs  very  elementary  to  the  operator  and 
therefore  provides  motivation  to  approach  the  problem  from  a  different  perspective.  In 
conclusion,  this  research  will  use  mouse  inputs  due  to  the  simplicity  of  the  setup  and  the 
ease  of  transition  between  it  and  a  touchscreen  in  future  development. 

2.2  Vision  Based  Navigation 

Vision  based  navigation  is  a  complex  task  which  has  not  been  fully  mastered. 

The  goal  of  vision  based  navigation  is  to  replace  human  vision  with  computer  vision 
which  sufficiently  performs  all  the  required  tasks.  Human  vision  is  an  amazing 
combination  of  the  eye  and  brain.  The  cornea  and  lens  of  the  eye  focuses  an  image  of  the 
outside  world  onto  a  light-sensitive  membrane  in  the  back  of  the  eye,  called  the  retina. 
The  retina  is  actually  part  of  the  brain  that  is  isolated  to  serve  as  a  transducer  for  the 
conversion  of  patterns  of  light  into  neuronal  signals  (Lewis,  2000). 
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Although  what  is  seen  is  a  major  influence  on  what  the  brain  perceives, 
psychologist  are  also  studying  “unconscious  inference,”  which  are  assumptions  the  brain 
believes  (Purves,  2007).  This  inference  strongly  influences  what  the  brain  believes  about 
vision  and  can  lead  to  visual  allusions.  This  problem  must  be  discussed  as  it  is  a  critical 
topic  when  replacing  pilots  with  UA.  The  replacement  of  human  pilot  vision  with  UA 
sensors  greatly  reduces  the  situational  awareness  of  a  ground  operator  yet  makes 
economical  sense  and  at  times  is  operationally  necessary.  A  review  of  research 
conducted  on  computer  vision  is  necessary  since  all  UA  carry  some  form  of  sensor 
equipment. 

2.2,1  Closely  Supervised  Reactive  Control  of  an  Uninhabited  Aerial  Vehicle, 

(Glassco,  1999) 

At  the  Air  Force  Institute  of  Technology  in  1999,  Captain  Glassco’s  thesis 
proposed  that  a  UA  operator  could  exercise  closely  supervised  reactive  control  by  target 
selection  from  on-board  camera  video.  The  flight  path  of  the  UA  was  determined 
autonomously  from  camera  gimbal  angles.  The  UA,  using  an  on-board  computer,  was 
given  autonomous  control  to  alter  its  flight  path  to  fly  towards  a  target  and  at  a  specified 
range,  loiter  over  the  target.  The  operator  had  two  modes  of  operation,  manual  and 
advanced.  In  the  manual  mode,  the  camera  operator  manually  tracked  the  target 
providing  continuous  updates  to  the  camera  angles.  In  the  advanced  mode  with  the  use  of 
INS  or  GPS,  the  aircraft  could  autonomously  determine  the  camera  angles  from  which  to 
fly  from  a  single  locked  position  which  would  leave  the  operator  free  to  look  for  other 
targets. 


19 


2.2.2  Study  of  Visual  Cues  for  Unmanned  Aerial  Vehicle  Waypoint 

Allocation,  (Trinh,  1999) 

A  1999  study  at  the  Massachusetts  Institute  of  Technology  focused  on  unmanned 
aerial  vehicles  with  humans  present  in  the  outer  supervisory  control  loop.  Visual  cues  for 
an  operator  station  were  evaluated  using  a  flight  simulator  with  the  goal  of  collecting  one 
minute  of  video  data  of  a  specified  target.  The  vehicle  contained  two  fixed  cameras,  one 
front-facing  and  one  side-facing.  Each  camera  had  a  vertical  FOV  (VFOV)  of  sixty 
degrees  and  horizontal  FOV  (HFOV)  of  thirty-six  degrees  and  each  was  tilted  downward 
forty-five  degrees.  The  simulation  assumed  a  point  mass  model,  constant  altitude  and 
velocity,  and  no  winds.  The  user  interface  was  a  graphical  display  and  mouse.  The  study 
concluded  that  the  side-viewing  camera  was  twenty-five  percent  more  effective  than  the 
forward  facing  camera  in  minimizing  total  time  required  to  accumulate  one  minute  of 
time  on  target.  However,  the  forward  view  was  preferred  for  target  locating  by  a  two  to 
one  ratio. 

2.2.3  A  New  Scheme  of  Vision  Based  Navigation  for  Flying  Vehicles  - 

Concept  Study  and  Experiment  Evaluation  (Jianchao,  2002) 

In  December  2002,  the  Signal  Processing  Faboratory  in  Singapore  proposed  a 
vision  based  navigation  scheme  for  long  range  navigation  in  which  the  main  navigation 
tool  was  a  camera,  complimented  by  an  altimeter.  The  fundamental  idea  was  to  infer 
inter-frame  3D  incremental  motion  of  the  camera  from  an  image  displacement  field 
embedded  in  two  consecutive  frames.  The  altimeter  was  an  aid  to  the  camera  in  order  to 
avoid  the  issue  of  uncertainty  in  scaling  factor,  so  that  the  metric  structure  of  motion 
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could  be  recovered  uniquely.  The  experiment  was  conducted  on  a  helicopter  with  INS 
and  GPS  also  installed  to  provide  a  baseline  position  measurement.  The  results  with  the 
camera  and  altimeter  were  far  from  the  accuracy  expected.  However,  when  gyroscope 
data  was  added  as  an  orientation  sensor,  the  position  error  was  greatly  reduced  from 
286.65  meters  to  4.9  meters. 

2.2.4  Vision-Based  Road-Following  Using  a  Small  Autonomous  Aircraft 

(Frew,  2004) 

In  January,  2004,  the  Center  for  Collaborative  Control  of  Unmanned  Vehicles  at 
the  University  of  California-Berkeley  designed  a  road-following  vision-based  control  for 
a  UA.  The  goal  of  the  work  was  to  enable  the  tracking  of  a  roadway  using  only  the 
natural  features  of  the  scene  and  no  additional  navigation  sensors.  The  computer  system 
would  detect  natural  features  of  the  scene  and  track  the  roadway  in  order  to  determine 
relative  yaw  and  lateral  displacement  between  the  aircraft  and  the  road.  The  problem 
required  detection  of  the  road  within  an  image  and  calculation  of  the  distance  of  the 
aircraft  away  from  it.  The  flight  of  the  aircraft  was  controlled  in  order  to  follow  the  road 
and  bring  the  cross-track  error  to  zero.  Hardware-in- the-loop  demonstrations  and  actual 
flight  tests  verified  the  performance  of  the  system  and  offered  encouraging  results. 

2.2.5  Vision-Only  Aircraft  Flight  Control  Methods  and  Test  Results 

(Proctor,  2004) 

In  August,  2004,  a  team  at  Georgia  Tech  successfully  developed  a  glider  which 
was  capable  of  flying  from  a  starting  point  to  an  ending  location  using  only  a  single 
vision  sensor.  This  research  differed  from  previous  work  in  that  it  utilized  only  vision  for 
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determining  the  position,  velocity,  attitude,  desired  flight  path,  and  required  control 
commands  in  an  outdoor  environment.  The  research  was  focused  on  the  idea  that  if  a 
human  pilot  can  visually  acquire  a  specified  destination  and  fly  to  it,  a  computer  should 
be  able  to  do  the  same.  The  system  has  the  advantage  of  having  a  simple  hardware 
configuration  making  it  a  low  cost  solution  for  an  autopilot  and  an  attractive  choice  for  a 
completely  isolated  back-up  system. 

2.2.6  Augmenting  UAV  Autonomy  -  Vision-Based  Navigation  and  Target 

Tracking  for  Unmanned  Aerial  Vehicles  (Lundington,  2006) 

In  September,  2006,  another  team  at  Georgia  Tech  demonstrated  a  vision-based 
navigation  system  on  an  unmanned  research  helicopter.  The  key  idea  was  to  incorporate 
visual  data  as  an  input  to  an  automatic  control  system.  The  visual  control  system  would 
aid  in  the  fonnation  of  a  navigation  solution,  identify  regions  of  interest  in  an  urban 
environment,  and  automatically  track  a  mobile  ground  target.  The  target  was  known  and 
assumed  to  be  within  view  of  the  camera.  The  navigation  solution  used  vision  and 
inertial  measurements  within  an  extended  Kalman  filter  to  estimate  the  vehicle  state.  An 
automated  search  system  allowed  the  UA  to  search  an  urban  region  for  a  particular 
building  and  then  identify  the  openings  to  the  building.  The  tracking  of  a  maneuvering 
ground  target  relied  upon  an  algorithm  integrated  with  the  camera  controller  which 
generated  a  waypoint  guidance. 

2.2.7  Fusion  of  Imaging  and  Inertial  Sensors  for  Navigation  (Veth,  2006) 

At  the  Air  Force  Institute  of  Technology  in  2007,  Major  Veth  presented  an 
approach  for  fusing  optical  and  inertial  sensors  for  robust,  self-contained,  passive,  and 
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autonomous  navigation.  The  motivation  behind  the  research  was  the  observation  of 
navigation  capabilities  in  animals.  Veth  developed  a  practical  model  of  sensors  and 
presented  a  transformation  technique  to  detect  features  within  an  image.  Coupling 
images  and  inertial  sensors  aids  the  statistical  feature  matching  function  which  in  turn  is 
used  to  estimate  navigation  trajectory  using  an  extended  Kalman  filter.  An  image-aided 
inertial  navigation  algorithm  was  tested  using  a  combination  of  simulation  and  ground 
tests.  Conclusions  pointed  out  limitations  of  the  Kalman  filter,  however  the  experimental 
results  “demonstrate  a  navigation  performance  improvement  of  at  least  two  orders  of 
magnitude  over  the  respective  inertial-only  solutions  (Veth,  2006).” 

2.2.8  White  Paper:  Analysis  of  Risk  Associated  with  Integrating  Sensors  on 

MAVs  (Garrett,  2007) 

At  the  Air  Force  Institute  of  Technology  in  2007,  Captain  Knowlan  and  Major 
Phillips  developed  a  risk  calculation  matrix  to  detennine  whether  engineers  can  design  a 
basic  MAV  which  is  effective  in  the  operational  context.  The  setup  consisted  of  four 
video  cameras;  two  were  mounted  boresight  on  the  front  and  two  were  depressed  thirty 
degrees  from  the  horizon  on  the  right  side.  Each  position  included  a  thirty  and  sixty 
degree  FOV  camera.  The  research  analyzed  at  what  range  the  user  would  be  able  to 
detect  (properly  name)  and  identify  (name  by  type  or  class  based  on  configuration)  a 
target  while  flying  at  100  m  and  50  m  and  how  much  time  the  user  would  have  to  react. 
The  definition  of  detect  and  identify  originated  from  the  National  Imagery  Interpretability 
Rating  Scale  (NIIRS).  The  maximum  range  a  target  could  be  detected  or  identified  was 
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calculated.  A  simulation  in  MATLAB  provided  a  visual  representation  to  link  the 
calculations  to  the  NIIRS  scale. 

Operational  scenarios  were  created  to  evaluate  the  MAV  in  flight.  At  100  m  in 
both  FOV,  the  front  camera  was  unable  to  identify  but  could  detect  the  target.  At  50  m  in 
both  FOV,  the  front  camera  was  able  to  briefly  identify  for  a  distance  and  then  detect  the 
target  for  the  remaining  distance.  For  the  side  camera  at  100  m,  the  thirty  degree  FOV 
camera  could  also  partially  identify  and  detect  the  target,  but  the  sixty  degree  FOV 
camera  could  only  detect  the  target.  At  50  m,  the  side  sixty  degree  FOV  camera  could 
also  identify  for  a  portion  of  the  time,  and  detect  the  rest  of  the  distance.  The  thirty 
degree  FOV  camera  at  50  m  was  able  to  identify  for  the  entire  range.  Having  the  side 
cameras  tilted  proved  a  clear  advantage  in  acquiring  the  target.  Also  noteworthy  was  the 
fact  that  the  time  on  target  could  be  increased  by  entering  a  controlled  turn  with  the  side 
camera  on  target.  Conclusions  stated  that  increasing  the  tilt  of  the  front  cameras  to  10-15 
degrees  reduced  the  required  distance  between  the  MAV  and  target  and  increased  the 
ability  to  identify  the  target. 

2.2.9  Summary  of  Vision  Based  Applications  to  Cursor-on-Target 

Although  vision  navigation  may  seem  like  a  simple  task,  recent  research  confirms 
it  is  still  a  topic  to  be  explored.  The  ability  to  track  a  target  without  the  need  of  GPS  and 
INS  is  an  attractive  goal,  not  only  in  cost  savings,  but  in  reduced  weight  requirements  and 
dependency  on  communication  and  navigation  satellites.  UA  dependant  on  vision 
guidance  alone  have  shown  practical  benefits  to  the  warfighter  on  the  ground  in  such 
systems  as  the  Marine’s  RQ-2B  Pioneer  and  the  Army’s  RQ-5A  Hunter  by  giving  the 
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ability  to  view  near  real-time  video  (United  States,  2005).  Vision  guidance  is  worth 
pursuing  in  even  greater  applications. 

The  technology  is  currently  available  to  incorporate  vision  feedback  into  the  logic 
of  onboard  automatic  control.  However,  the  challenge  facing  the  military  will  be  to  tailor 
this  technology  to  the  needs  of  the  specific  Services.  One  UA  cannot  perform  all 
missions  because  of  various  sensor  capabilities  and  the  varying  requirements  of  military 
missions.  This  research  will  focus  on  missions  that  require  an  operator  in  the  control 
loop  at  certain  times  because  even  those  missions  labeled  “dull”  can  at  times  require  the 
element  of  human  judgment.  The  goal  will  be  to  incorporate  the  lessons  learned  from 
previous  research  to  develop  a  simple  way  for  operators  to  control  UA  using  vision 
feedback  to  the  ground  control  station. 
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III.  Problem  Setup  and  Definitions 


This  chapter  explains  the  basic  building  blocks  necessary  to  develop  COT. 

3.1  Assumptions  for  COT  Development 

Before  beginning  the  task  of  developing  a  new  controller,  many  assumptions  must 
be  made  to  simplify  the  process.  The  following  assumptions  were  made  for  this  research: 

1 .  The  UA  is  modeled  as  a  point  mass. 

2.  Flight  is  at  a  constant  altitude. 

3.  The  UA  has  adequately  designed  inner  control  loops  that  can  hold  a  constant 

altitude  and  provide  a  commanded  turn  rate  and  airspeed  acceleration  (within  the 
limits  of  the  UA). 

4.  The  UA  has  an  electro-optic  camera  with  a  fixed  angle  and  geometry. 

5.  The  target’s  motion  is  within  the  UA’s  flight  envelope  (i.e.  possible  to  track 
the  target). 

6.  Wind  speed  is  negligible. 

7.  A  human  controller  is  required  to  be  a  part  of  the  control  loop  at  some  point  in 
order  to  accomplish  the  UA’s  mission. 

These  assumptions  provide  a  simple,  yet  adequate  model  to  develop  and  test  the 
feasibility  of  COT.  Assumption  seven  is  the  driving  motivation  behind  COT  controller. 
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3.2  Sensor  Information 


Since  UA  depend  on  onboard  sensors  to  provide  situational  awareness  to  the 
ground  station  operator,  the  sensor’s  information  is  an  important  element  to  understand. 
The  sensor  footprint  is  three-dimensional  infonnation  being  translated  onto  a  two- 
dimensional  screen.  Since  an  in-depth  study  has  not  been  conducted  on  electro-optical 
sensors  for  this  research,  only  a  basic  explanation  will  be  provided.  The  sensor  footprint 
depends  at  a  minimum  on  the  sensor’s  specifications,  mounting  geometry,  and  vehicle’s 
position,  altitude,  and  attitude.  Various  recommendations  on  mounting  geometry  have 
been  proposed  by  previous  research,  yet  each  UA  will  have  different  geometry  based  on 
mission  needs  and  the  airspeeds  flown. 

The  geometry  of  the  sensor  footprint  can  be  described  using  three  different  views: 
side-vertical  view  (see  Figure  6),  top-down  view  (see  Figure  7),  and  frontal  view  (see 
Figure  9)  as  described  in  detail  in  (Rufa,  2007).  For  this  research,  only  the  vertical  field 
of  view  (VFOV),  as  shown  in  Figure  6,  and  horizontal  field  of  view,  as  shown  in  Figure 
9,  were  used.  Figure  7  shows  the  trapezoidal  footprint  that  is  translated  to  a  rectangular 
display  as  depicted  by  Figure  8. 


Figure  6.  Side-Vertical  view  of  UA  sensor  beam  (Rufa,  2007) 
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Line  2 


Figure  7.  Top-down  view  producing  trapezoidal  footprint  of  UA  sensor  beam  (Rufa,  2007) 


Figure  8.  Graphical  User  Interface  Display  with  trapezoidal  lines  transformed  to  straight  lines 


Figure  9.  Frontal  view  of  UA  sensor  beam  (Rufa,  2007) 
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The  sensor  footprint  can  vary  depending  on  the  sensor’s  FOV.  From  high 
altitudes,  the  footprint  will  be  more  rectangular  based  upon  a  steep  depression  angle.  At 
lower  altitudes  with  shallower  depression  angles,  the  footprint  has  a  trapezoidal  shape  yet 
it  is  displayed  on  a  rectangular  display.  This  can  be  visualized  by  imagining  three 
separate  lines,  two  at  the  edges  of  the  display,  and  one  down  the  middle.  The  outside 
lines  are  in  reality  the  outside  diagonal  lines  of  the  trapezoidal  footprint,  with  the  only 
true  straight  line  being  the  middle  line  (see  Figure  7).  All  information  that  a  sensor 
collects  along  a  line  in  a  given  direction  is  displayed  in  vertical  straight  lines  on  the 
display  (see  Figure  8). 

The  display  of  a  trapezoidal  footprint  onto  a  rectangle  display  aids  in  the 
development  of  COT  in  the  following  way.  The  placement  of  a  cursor  on  a  display  may 
correspond  in  distance  from  a  few  hundred  yards  to  a  few  miles  from  the  UA.  For  lateral 
control,  this  seemingly  would  pose  a  problem  as  placement  of  the  cursor  in  the  distance 
would  turn  a  UA  more  than  needed.  But  since  all  vertical  lines  are  in  reality  diagonal 
lines  of  the  trapezoidal  footprint,  the  horizontal  position  of  the  cursor  on  the  display  is 
proportional  to  the  angle  between  the  UA’s  heading  and  the  line-of-sight  angle  to  the 
heading  marked  by  the  cursor  (see  Figures  7  and  8). 

3.3  Electro-Optical  Target  Acquisition  Process 

An  electro-optical  (EO)  system  is  one  in  which  “optics  is  used  to  collect  photons 
in  the  optical  range  of  the  electro-magnetic  spectrum  and  convert  them  into  electrons  that 
can  be  processed  as  electronic  signals  for  a  variety  of  useful  purposes  (Minor,  2002).” 
Such  purposes  are  determined  by  the  recipient  of  the  information,  and  the  human 
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observer  is  still  the  principle  recipient  of  modern  EO  systems.  Thus,  despite 
technological  advances,  the  ground  operator  is  the  prime  decision  maker  and  target 
acquisition  device  for  most  UAS  using  EO  imaging  systems.  It  is  the  ground  operator’s 
ability  to  interpret  and  use  the  information  being  presented  by  the  EO  system  which  is  the 
critical  element  in  the  performance  of  the  overall  system. 

The  electro-optical  acquisition  process  is  a  complex  series  of  events  depending  on 
the  order  of  many  real  world  variables.  The  key  variables  are  target  and  background,  the 
atmosphere,  the  electro-optical  system,  the  display  system,  and  human  performance 
limitations  (see  Figure  10).  These  variables  combine  to  fonn  an  overall  probability  of 
target  detection,  recognition,  identification,  and  classification.  Since  the  goal  of  the 
acquisition  process  is  to  produce  a  system  that  meets  operational  requirements,  one  must 
understand  the  impact  of  the  key  variables  in  the  overall  testing  process  (Minor,  2002). 


Figure  10.  Elements  of  Target  Acquisition  Process  (Minor,  2002) 

Although  the  key  variables  can  be  broken  into  an  even  longer  list,  eventually  one 
must  weigh  the  relative  importance  of  each  in  order  to  evaluate  the  overall  system.  EO 
system  designers  have  concluded  that  the  major  inputs  to  the  performance  of  EO  sensors 
are  dependant  on  both  target  and  background  and  the  human  observer’s  performance. 
Since  the  target  and  background  can  be  pre-planned  to  optimize  acquiring  the  target,  in 
the  end  it  is  the  human  observer’s  performance  which  is  the  limiting  factor  when 
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evaluating  an  EO  system,  and  in  particular,  the  UAS  system.  As  a  result,  the  interface 
which  a  ground  operator  uses  is  of  critical  importance  when  designing  any  UAS.  Using 
the  applicable  “EO  system  flight  tester  fundamental  disciplines”  presented  in  Minor’s 
paper  as  a  baseline,  any  UAS  designer  and  flight  tester  should  develop  a  solid  grasp  of 
the  disciplines  of  optics,  display  systems,  and  human  search  processes/visual 
psychophysics  (Minor,  2002). 


3.4  Vehicle  Equations-of-Motion 

The  equations  of  motion  form  the  nucleus  of  the  simulation,  which  is  presented  in 
the  next  chapter.  Although  relatively  elementary,  the  point-mass  UA  model  presented 
below  was  deemed  sufficient  for  the  research.  Follow-on  research  will  no  doubt  require  a 
more  in-depth  analysis  of  the  equations.  To  begin,  differential  equations  were  created  as 
follows: 
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where  VTAS  is  true  airspeed,  %  is  heading  angle,  y  is  flight  path  angle,  N  is  north 

position,  E  is  east  position,  and  h  is  altitude.  w, ,  u: ,  and  u3  are  the  acceleration,  turn- 

rate,  and  altitude  commands  from  the  COT  controller  to  the  UAS.  The  north  and  east 
components  of  the  wind  are  obtained  from: 

WN  =W  cos  zw 


WE=Wsmzw. 


where  W  is  wind  speed  and  xw  is  wind  heading  angle. 


The  equation  for  the  UAS’s  hank  angle  which  drives  the  camera  model  in  the  simulation 


is: 
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However,  given  the  assumptions  of  constant  altitude  and  no  wind,  these  simplified  to: 
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3.5  Chapter  Summary 


In  summary,  this  chapter  gave  the  background  necessary  for  COT  development. 
The  assumptions  used  in  the  simulation  were  listed,  sensor  information  and  the  electro¬ 
optic  target  acquisition  process  were  explained,  and  the  vehicle  equations-of-motion  were 
presented. 
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IV.  UAS  Simulation  for  Cursor-on-Target  Control  Development 


4.1  Simulation  Software 

All  research  was  performed  in  computer  simulation  using  the  MATLAB  and 
Si mul ink  program  (The  MathWorks,  2004),  and  the  Aviator  Visual  Design  Simulator 
(AVDS)  (Rasmussen,  2005).  The  controller  was  first  written  in  Simulink  and  then 
connected  to  AVDS  to  verify  realistic  responses,  and  additionally  provides  a  real-time 
operator-in-the-loop  capability. 

4.1.1  Simulink 

Simulink  is  a  software  package  integrated  into  the  computer  program  MATLAB. 
The  purpose  of  Simulink  is  the  modeling,  simulating,  and  analyzing  of  both  linear  and 
non-linear  dynamic  systems.  It  is  an  interactive  graphical  environment  with  a 
customizable  set  of  block  libraries  that  allow  models  to  be  built  as  block  diagrams. 
Complex  designs  can  be  broken  into  hierarchical  top-down  type  organization.  Simulink 
proved  useful  in  this  research  because  one  could  experimentally  pose  a  problem,  model 
it,  and  analyze  the  results  with  relative  ease.  Simulink  also  proved  valuable  due  to  its 
ability  to  interface  with  another  simulation  program,  in  particular  AVDS,  which  provided 
real-time  simulated  visualization  (i.e.  simulated  EO  imagery). 

4.1.2  Aviator  Visual  Design  Simulator  (AVDS) 

The  Aviator  Visual  Design  Simulator  (AVDS)  is  a  highly  detailed  personal 
computer  based  flight  simulator  used  in  aerospace  research  and  development.  It  was 
designed  as  a  joint  project  between  Artificial  Horizons,  Inc.  and  the  United  States  Air 
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Force.  AVDS  allows  researchers  to  visualize  flight  dynamics  and  flight  controls  and 
displaying  aircraft  orientation  (See  Figure  11).  These  features  allowed  realistic  testing  of 
the  COT  controller  developed  in  Simulink.  At  the  same  time,  it  is  simple  to  use  because 
Simulink  blocks  are  available  to  connect  AVDS  to  MATLAB/Simulink  (Rasmussen, 
2005).  An  additional  characteristic  of  interest  for  follow-on  research  is  the  ability  for 
AVDS  to  interact  with  external  hardware  inputs  and  have  hardware-in-the-loop  simulator 
capability  (Dugan,  2006). 


Figure  11.  AVDS  display  showing  sensor  view 

4.2  UAS  Simulation 
4.2.1  Overview 

The  purpose  of  the  UAS  simulation  is  to  create  a  realistic  model  of  the  UAS  so 
the  COT  concept  can  be  developed  in  a  realistic  setting  and  transferred  to  real  world 
application.  The  UAS  simulation  can  be  divided  into  four  main  subsystems:  UA  model, 
Camera  Model,  Mouse  Position  from  Graphical  User  Interface  (GUI),  and  the  COT 
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Control  System  (see  Figure  12).  Additionally,  to  test  the  ability  of  the  UAS  to  track  a 
ground  target  using  the  COT  controller,  a  target  system  was  created.  Simulink  was  the 
tool  used  to  simulate  the  UA,  the  COT  Control  System,  and  the  target.  AVDS  was  the 
tool  used  to  simulate  the  Camera  Model  and  used  to  extract  a  Mouse  Position  from  the 
GUI. 


from  Ground  Station 


Modem  to  Autopilot 


Figure  12.  COT  Concept  with  realistic  subsystem  divisions  in  Simulink 


4.2.2  Cursor-on-Target  Control  System 

The  preliminary  COT  control  system  developed  tracks  the  target  by  use  of  both 
Heading  Control  and  Velocity  Control.  The  COT  controls  heading  by  commanding  the 
d  y 

UA’s  turn  rate,  — — ,  and  it  controls  velocity  by  commanding  the  UA’s 
dt 

dV 

acceleration,  — — .  The  required  inputs  for  the  COT  controller  are  the  X  and  Y  position 
dt 

of  the  mouse  in  the  camera  display  GUI,  the  camera  geometry,  and  the  UA’s  bank  angle. 
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The  turn  rate  controller  used  for  heading  control  is  based  on  missile  guidance,  and 
a  few  options  were  considered  (Garnell,  1980).  First  was  pure  pursuit  control,  where 
d  y 

— —  =  K a/  and  the  commanded  turn  rate  is  proportional  to  the  heading  error,  aj  . 
dt 


Second  was  proportional  navigation,  where 


dx, 


cmd 


dt 


Ka%  and  the  commanded  turn  rate 


is  proportional  to  the  rate  of  change  of  the  heading  error.  Finally,  the  controller 
implemented  was  a  Dynamic  Inversion  Turn  Rate  Controller  used  in  (Burns,  2007)  where 
d  y 

— —  =  K  tan(A^)  .  This  turn-rate  controller  has  properties  of  both  of  the  previous  turn 
dt 

rate  controllers.  Its  performance  is  similar  to  the  proportional  navigation  controller,  but 
like  the  pure  pursuit  controller,  the  commanded  turn  rate  is  a  function  of  the  heading 
error.  As  was  explained  in  the  Sensor  Information  section,  the  heading  error,  a/  ,  is 
proportional  to  the  x-position  of  the  cursor  from  the  center  of  the  display,  aXgui  ,  which 


HFOV 

is  obtained  from  the  GUI.  This  can  be  written  as  a %  = - a Xgui  ,  where  HFOV  is 


the  horizontal  field  of  view  of  the  UA  sensor. 


For  velocity  control,  an  acceleration  controller  is  used.  Similar  to  the  pure  pursuit 


dV 

type  controller  for  turn  rate,  the  acceleration  controller  is  — <=^~  =  KvaV  and  the 

dt 


commanded  acceleration  is  proportional  to  the  velocity  error,  aV  .  By  implementing  the 
controller  using  the  camera  GUI,  let  aV  =  K  a  Ygui  ,  where  a  Ygui  is  the  y-position  of  the 

cursor  from  the  center  of  the  display.  Then,  the  final  form  of  the  equation  is 
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— ^MD  =  KvKvaY  =  K a  Ygui  .  In  summary,  this  literally  means  that  a  turn  rate  will  be 

commanded  if  the  a  Xgui  is  not  zero  and  an  acceleration  will  be  commanded  should 
a Ygui  not  equal  zero. 


4.2.3  Mouse  Position  from  Graphical-User-Interface 

The  GUI  visually  displays  the  mouse  position  to  the  operator.  Simulated  camera 
video  is  generated  by  AVDS  and  displayed  in  the  graphical-user-interface  (GUI)  along 
with  the  mouse  position.  The  only  AVDS  signals  forwarded  to  the  Simulink  COT 
Control  System  are  mouse  cursor  positions.  Since  the  equations  for  COT  require  as 
inputs  the  change  in  horizontal  direction,  aXgui  ,  and  vertical  direction,  a Ygui  ,  one  must 

first  map  the  camera  display  from  -1  to  +1  in  both  directions.  With  wings  level,  which  is 
the  simplest  aircraft  orientation,  the  inputs  to  the  COT’s  turn  rate  and  acceleration  control 


equations  are  a/ 


HFOV 


a X GUj  and  aV 


VFOV 


.Y 


GUI  • 


The  reference  fields  are 


divided  in  half  due  to  the  center  of  the  fields  being  the  reference  zero. 


When  the  aircraft  banks,  aIot  and  a  Ygui  must  be  recalculated  to  include  the 
bank  angle  cp  and  new  terms  a Hgu1  and  aVgui  are  created  to  include  the  position  of  the 
mouse  along  the  horizon  and  vertically  from  the  horizon  (See  Figure  13).  The  former 


equations  become  ax  = 


HFOV 


a HGU1  and  aV  = 


VFOV 


.V 


GUI 


where  a Hgui  =aXgui  cos((p)  -aYgu1  sin(V/?) 


and  aVgui  aXgui  sin(^>)  +aYgui  cos {(p) . 
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This  allows  the  turn  rate  and  acceleration  to  remain  constant  if  correctly  tracking  a  target 
while  the  UA  is  banked  even  though  the  X  and  Y  coordinates  are  not  mapping  to  zero 
and  zero  respectively. 


Figure  13.  COT  following  the  target  during  a  banked  turn 

4.2.4  Unmanned  Aircraft  Model 

The  UA  model  contains  equations  which  drive  the  camera  model  in  the  AVDS 
simulator.  The  equations  explained  in  the  Vehicle  Equations-of-Motion  Section  form  the 
backbone  of  the  model  and  are  included  in  an  embedded  MATLAB  function.  There  are 
seven  inputs  to  the  model.  The  first  three  inputs,  ux  — >  it-,  include  the  turn  rate  command 

and  acceleration  command  outputs  from  the  COT  controller  and  the  commanded  altitude. 
Note  that  the  turn  rate  command  and  acceleration  command  are  passed  through  first-order 
low-pass  filters  to  account  for  some  actuator  dynamics.  The  next  four  inputs,  x,  — »  x4  are 

actually  states  extracted  from  the  model  itself  as  a  feedback  loop  once  the  equations  have 
been  integrated  and  include  the  aircraft  velocity,  heading,  and  north  and  east  position. 
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The  UA  model  outputs  six  variables  that  are  sent  to  AVDS  to  drive  the  camera 
simulator.  Two  outputs,  bank  angle  and  altitude,  are  sent  directly  to  AVDS  (see  Figure 
14).  The  other  four  outputs,  x,  — >•  x4 ,  include  acceleration,  rate  of  heading  change,  and 

position  rate  of  change  in  north  and  east  directions.  These  outputs  are  fed  through  an 
integrator  before  flowing  to  AVDS  and  also  being  fed  back  as  inputs. 


Figure  14.  UA  Model  in  Simulink 

4.2.5  Camera  Model 

The  simulator  AVDS  served  as  the  camera  model  for  the  research.  AVDS  calls 
for  twenty-five  inputs  and  produces  sixteen  outputs.  For  this  simulation,  the  six  inputs  of 
altitude,  bank  angle,  velocity,  heading  angle,  and  north  and  east  position  were  used  with 
the  remaining  nineteen  inputs  left  blank.  The  outputs  used  were  simply  the  cursor 
position  of  the  mouse  in  the  GUI,  which  correspond  to  aXgui  and  a Ygu1  .  In  the  AVDS, 
the  camera  view  can  be  manually  adjusted  to  give  different  VFOV,  HFOV,  and 
depression  angle. 
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4.2.6  Target  Model 


The  target  model  contained  an  embedded  MATLAB  function  with  the  same 
equations  of  motion  used  for  the  UAS.  The  output  of  the  target  model  was  connected  to 
the  AVDS  simulator  using  the  AVDS  Network  Send  Connection  (see  Figure  15).  The 
inputs  are  all  adjustable  to  give  flexibility  in  testing.  The  outputs  also  include  flexibility 
in  the  actual  position  of  the  aircraft. 


Figure  15.  Target  Model  in  Simulink 
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4.3  Chapter  Summary 


This  chapter  explained  the  computer  simulation  used  to  develop  COT.  MATLAB 
and  Simulink  were  integrated  with  AVDS  to  provide  a  realistic  environment  in  which  to 
demonstrate  an  operator-in-the  loop  capability.  The  UAS  simulation  included  five  main 
elements:  UA  model,  camera  model,  mouse  position  from  GUI,  COT  control  system  and 
target  model.  The  key  component  of  the  system,  the  COT  controller,  commands  the 
UAS’s  acceleration  and  turn-rate,  which  enables  an  operator  to  track  a  target  by 
positioning  the  mouse  in  the  camera  display  GUI. 
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V.  Proposed  UAS  Operator  Rating  Scale 


5.1  Cooper-Harper  Scale  Background 

Currently,  manned  aircraft  test  programs  depend  on  the  pilot’s  subjective  rating  of 
an  aircraft’s  handling  qualities.  Handling  qualities  are  defined  as  “those  qualities  or 
characteristics  that  govern  the  ease  and  precision  with  which  the  pilot  is  able  to  perform 
the  task  required  in  support  of  the  aircraft’s  role  (Liebst,  2006).”  The  rating  scale 
universally  accepted  by  test  pilots  to  measure  handling  qualities  is  the  ten  point  Cooper- 
Harper  scale.  The  scale  is  broken  up  into  levels;  Level  1  is  satisfactory  (1-3.5),  Level  2 
is  adequate  (4-6.5),  and  Level  3  is  tolerable  (7-9),  and  a  value  of  ten  does  not  allow  the 
plane  to  be  flown  again.  The  scale  was  created  to  be  pilot-oriented  and  can  be  used  when 
one  does  not  know  how  to  exactly  quantify  the  results  of  a  test.  The  drawbacks  to  the 
scale  are  the  lack  of  design  guidance  and  the  need  for  compliance  to  standards  on  the  part 
of  the  pilot  (Hodgkinson,  1999). 

5.2  Lack  of  current  UA  rating  scales 

The  application  of  a  scale  similar  to  the  Cooper-Harper  Rating  scale  to  UA 
development  is  a  logical  progression  due  to  the  dependence  on  the  ground  operator  as  the 
key  element  of  semi-autonomous  UA.  In  COT,  the  task  is  similar  to  the  handling 
qualities  test  of  a  manned  aircraft  pilot.  Although  the  use  of  a  scale  similar  to  the 
Cooper-Harper  rating  scale  for  UA  is  proposed  for  this  research,  use  of  such  a  scale  is  not 
currently  a  published  widespread  practice  for  rating  UAS  performance.  The  only 
published  article  on  similar  UA  rating  scales  was  written  in  1992  by  the  Naval  Air 
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Warfare  Center.  In  that  article  titled  “Flying  Qualities  of  Remotely  Piloted  Vehicles,”  the 
authors  proposed  an  initial  attempt  at  developing  a  technique  to  facilitate  the 
quantification  of  UA  flying  qualities,  but  did  not  propose  using  the  Cooper-Harper  scale 
(Breneman,  1992).  By  incorporating  a  standard  rating  scale  for  UAS,  designers  could 
incorporate  proven  handling  qualities  from  the  earliest  stage  of  design.  Such  a  rating 
scaled  would  have  provided  a  reference  point  for  continuity  between  previous  research 
such  as  the  Army  Research  Institute’s  MAV  and  the  current  project.  Without  a  scale  on 
which  to  base  results,  the  article  on  the  Army’s  research  is  generally  helpful,  but  no 
specific  test  results  can  be  obtained. 
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5.3  UAS  Rating  Scale 


The  present  research  proposes  an  updated  Cooper-Harper  rating  scale  specifically 
applicable  to  handling  qualities  during  tracking  tasks  (see  Figure  16).  The  only 
modification  to  the  Cooper-Harper  scale  are  two  blocks  adopted  from  the  Pilot  Induced 
Oscillation  rating  scale  which  concern  the  operator  entering  the  control  loop  and 
initiating  tight  control  while  in  the  loop  (see  Figure  17).  Although  proposed  by  a  military 
pilot  with  only  academic  knowledge  of  the  Cooper-Harper  rating  scales,  this  scale  will 
hopefully  provide  a  basis  for  quantifiable  results  when  testing  the  preliminary  COT 
design  concept  and  provide  a  starting  point  of  dialogue  between  engineers  and  UA 
ground  operators. 


Figure  16.  Handling  Qualities  during  Tracking  Diagram  (Liebst,  2006) 
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Figure  17.  Proposed  UAS  Operator  Rating  Scale 
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VI.  Demonstration  and  Test  Results 


6.1  Demonstration 

The  purpose  of  the  demonstration  was  to  show  that  the  development  of  a  COT 
controller  actually  works  in  a  virtual  environment.  After  many  trial  runs  and  much 
troubleshooting,  the  computer  model  operated  as  expected  (see  Figure  18).  With  the 
UAS  model  flying  at  a  predetermined  altitude,  lateral  movement  as  directed  by  the  cursor 
acts  as  a  turn  rate  controller  for  the  aircraft  model.  The  velocity  is  adjusted  by 
acceleration  control  for  cursor  movement  in  the  vertical  plane.  Although  it  may  be 
considered  an  elementary  model,  it  paves  the  way  for  future  research  and  a  more 
complicated  demonstration. 


Figure  18.  Demonstration  of  COT  with  Target  Model  in  view 
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6.2  Test  Plan 


Flight  testing  is  required  at  the  end  of  any  aircraft  design  process  to  validate  and 
refine  the  design.  Testing  determines  whether  the  aircraft  and  crew  can  safely 
accomplish  the  intended  mission.  Aircraft  changes  can  be  made  as  a  result  of  flight  tests, 
but  the  tests  must  be  planned  correctly  or  an  endless  cycle  of  changes  will  be  made, 
potentially  leading  to  program  failure.  Although  the  current  research  does  not  include 
actual  flight  testing,  any  aircraft  design  requires  a  number  of  preliminary  ground  tests 
prior  to  the  first  flight.  Including  a  background  and  framework  for  testing  will  be 
beneficial  for  both  the  current  computer  simulation  and  any  future  work  in  this  area  of 
research  (Kimberlin,  2003). 

When  planning  any  test,  one  must  determine  the  purpose  and  objectives  of  the  test 
program.  For  the  current  research,  the  purpose  of  testing  is  to  detennine  if  the  developed 
COT  controller  performs  correctly  in  realistic  scenarios.  Next,  one  should  design  the  test 
and  use  methods  to  accomplish  the  stated  objectives  of  demonstrating  the  feasibility  of 
COT  by  testing  various  real-world  scenarios.  If  possible,  one  should  review  previous 
tests  for  lessons  learned  and  design  the  test  for  a  specific  task.  In  this  research,  no  similar 
research  was  available  upon  which  to  model  testing.  The  specific  task  is  to  keep  a 
ground  target  in  view  of  the  camera.  Throughout,  one  should  conduct  brainstorming 
sessions  in  order  to  create  original  ideas.  In  conducting  the  test,  a  group  of  five  to  ten 
operators  will  give  good  results  (Kimberlin,  2003). 

Some  techniques  have  been  developed  in  designing  a  test.  First,  pre-simulator 
briefing  requires  some  forethought  so  inadvertent  bias  is  not  introduced.  Second, 
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configurations  should  be  randomized  but  begin  with  easier  configurations.  At  the  same 
time,  one  should  expose  the  operator  to  the  extremes  of  the  configuration  early  on  in  the 
experiment  in  order  to  see  contrast  in  the  ratings.  Lastly,  operators  should  be  given  a 
long  look  at  the  task  by  allowing  the  configuration  to  be  flown  for  two  minutes  and  then 
repeated  before  an  evaluation  is  given  (Hodgkinson,  1999). 

After  searching  for  set  procedures  to  follow  in  an  aircraft  test,  no  set  standard  was 
found  because  each  test  is  unique.  However,  an  accepted  technique  was  proposed  as 
follows.  First,  one  should  define  the  desired  and  adequate  bounds  of  the  test.  Second, 
the  task  should  be  performed.  Third,  the  operator  should  comment  extemporaneously 
during  the  task  to  collect  as  much  infonnation  as  possible.  Fourth,  the  operator  should 
assign  an  initial  rating  using  the  proposed  UAS  operator  rating  scale.  Fifth,  detailed 
comments  should  be  written  using  provided  comment  cards.  Finally,  the  formal  rating 
should  be  given  to  the  operator  and  only  then  the  quantitative  performance  of  the  flight  is 
revealed  to  the  operator  (Kimberlin,  2003). 

Such  steps  in  the  planning  and  implementation  of  the  test  can  produce  reliable  and 
repeatable  results.  One  key  element  when  debriefing  operators  is  to  question  why  the 
pilots  assigned  a  particular  rating.  Although  the  majority  of  operators  will  most  likely  be 
within  one  point  each  other,  a  varied  response  should  not  be  discounted  as  it  may  uncover 
an  error  in  the  model  to  be  tested  (Kimberlin,  2003). 
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6.2.1  Test  Plan  Scenarios 


Due  to  the  preliminary  nature  of  the  research,  the  test  scenarios  were  kept 
straightforward  and  simple.  At  the  same  time,  the  scenarios  covered  all  realistic  cases  for 
a  moving  target.  The  case  of  a  stationary  target,  a  target  that  stops  intermittently,  or 
travels  slower  than  the  UA  was  not  covered  and  will  be  a  topic  for  continued  research.  In 
addition,  this  research  only  pertains  to  a  target  that  is  in  front  of  the  UA  and  circling 
patterns  are  not  discussed. 

Each  operator  was  given  eight,  two  minute  scenarios  in  which  to  operate  COT 
(see  Table  2).  The  first  scenario  was  a  demonstration  to  educate  the  operator  on  the  use 
of  COT.  In  the  second  scenario,  the  operator  was  instructed  to  simply  keep  the  cursor  on 
the  target,  as  the  COT  control  name  implies.  In  the  remaining  scenarios,  the  goal  was  to 
lead  the  target  with  the  cursor  in  order  to  keep  the  target  in  the  middle  of  the  screen.  This 
goal  was  tested  in  scenarios  three  through  six  with  different  heading  and  airspeed 
variations. 

Table  2.  Scenarios  Flown  to  Evaluate  COT 


1.  Demonstration 

2.  Cursor-on-Target 

3.  Constant  Heading  and  Airspeed 

4.  Varying  Heading  and  Airspeed _ 

5.  Constant  Heading,  Varying  Airspeed 

6.  Varying  Heading,  Constant  Airspeed 

7.  Adjusted  Gain  in  Scenario  #  4 _ 

8.  Adjusted  Gain  in  Scenario  #  4  with 
Wings  Level  Flight 
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The  last  two  scenarios  had  adjusted  gains,  with  the  last  scenario  restricted  to 
wings  level  flight  (see  Table  2).  The  reason  for  changing  the  gains  was  twofold;  first,  to 
see  if  such  gains  influenced  the  ability  to  track  the  target  and  second,  to  produce  a 
different  rating  number  to  verify  that  the  rating  scale  differentiated  between  various 

d  y 

scenarios.  The  gains  were  adjusted  for  the  equations  of  — —  =  K  tan(A^)  and 

dt 

dV  - 

— =  KaY...,.  described  in  section  4.2.2  .  In  the  initial  scenarios,  K  —  0.1  and 

dt  GUI 

K  - 125  which  after  trial  and  error  were  adjusted  to  K  —  5  and  K  =  10.  The  adjustment 
of  K  caused  the  most  influence  on  operator  ratings  due  to  the  instability  it  caused  in 
scenario  seven  when  the  cursor  was  adjusted  in  the  vertical  plane. 

Although  subjective  and  conducted  by  operators  with  minimal  rating  experience, 
the  test  did  give  a  baseline  to  at  least  discuss  COT.  Overall,  the  operators  rated  the  COT 
relatively  well.  For  the  four  basic  scenarios,  only  two  operators  gave  ratings  outside  of 
Level  1  (see  Figure  19).  Of  particular  note  are  scenarios  four,  seven  and  eight  of  the 
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Figure  19.  Operator  Rating  Results  for  Various  Scenarios 
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target  with  varying  heading  and  airspeed  (See  Figure  20).  When  the  gain  values  were 
increased  in  scenario  seven,  all  operators  rated  the  COT  controller  worse  and  only  one 
operator  rated  it  Level  1.  Yet,  with  the  same  increased  gains  but  wings  level  flight,  two 
of  the  three  operators  tested  rated  COT  equal  or  better  than  with  the  beginning  gains 


Operator  Ratings  for  Varying  Heading  and  Airspeed  Scenarios 


♦  Operator  4 
■  Operators 
4  Operators 


Tests 


Figure  20.  Operator  Rating  Results  for  Varying  Heading  and  Airspeed  Scenarios 

which  included  bank  angle.  From  this  simple  task,  minimizing  bank  angle  seemed  to 
correlate  with  better  tracking  by  the  operator  and  may  be  worthy  of  continued  research. 
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VII.  Recommendations  and  Conclusions 


7.1  Recommendations  for  Future  Work 

Although  this  research  produced  a  preliminary  COT  control  system,  much 
improvement  can  be  made  to  bring  the  concept  closer  to  an  operational  reality.  In 
particular,  the  following  suggestions  are  made  for  future  work: 

•  A  more  sophisticated  COT  control  law  should  be  developed  in  order  to 
respond  better  to  target  movements  (e.g.  PID). 

•  Wind  influence  on  the  equations  of  motion  must  be  considered  as  it  has  a 
direct  impact  on  small  UA  flight. 

•  Development  of  more  in-depth  equations  of  motion  should  be  made  so 
altitude  can  become  a  variable. 

•  More  realistic  test  scenarios  should  be  developed  for  stopped,  stop  and  go,  and 
slow  moving  targets. 

•  Camera  views  as  described  in  previous  research  should  be  investigated, 
particularly  side-facing  cameras. 

•  This  research  did  not  include  the  opinion  of  any  real-world  UA  operators. 

Such  inputs  would  be  valuable  for  both  the  test  plan  and  COT  development. 

•  One  should  consider  additional  measures  of  performance  such  as  average  and 
maximum  distance  of  the  target  from  center,  and  the  operator’s  ability  to  keep 
the  target  within  defined  bounds. 
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•  The  COT  should  be  tested  with  hardware-in-the-loop,  including  live  video 
testing,  and  then  be  flown  on  a  UA  to  validate  that  COT  works  operationally. 

7.2  Additional  Comments 

Since  camera  optics  is  a  vital  portion  of  UA,  future  researchers  interested  in  this 
field  should  consider  taking  an  optics  class  to  better  understand  sensor  equipment.  After 
completing  this  research,  the  author  thinks  it  would  be  beneficial  to  institute  a 
standardized  UA  test  operator  training  program  similar  to  the  various  Services  Test  Pilot 
schools. 

7.3  Conclusions 

The  goal  of  this  research  was  to  develop  and  evaluate  a  COT  control  system  in 
order  to  decrease  operator  workload  during  high  task  phases  of  flight  and  to  develop  a 
method  of  evaluating  a  COT  control  system.  The  development  goal  has  been 
accomplished  by  creating  a  combined  Simulink-AVDS  simulation  which  included  a  UAS 
and  target  model.  In  the  simulation,  a  COT  controller  was  developed  and  the  concept 
was  demonstrated.  The  Operator  Rating  Scale  was  demonstrated  as  a  tool  to  evaluate 
UAS  guidance  and  control  and  the  results  proved  favorable.  This  provided  a  way  to 
compare  current  research  to  future  research  based  upon  a  time  tested  model  rating  scale. 
Hopefully,  this  research  will  pave  the  way  for  future  exploration  of  ways  to  control  UA 
that  do  not  require  in-depth  training  for  the  operator! 
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Appendix  A  :  Rating  Scales 


Cooper  Harper  Rating  Scale 
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Pilot  Induced  Oscillation  Rating  Scale 


PIO  Rating  Scale 


d°  \ 

Undesirable 
Motions  Tend  to 
Occur?  / 


Is  N. 
Task  \ 
Performance 
'.Com  promised? 


Causes 

Oscillations? 


Description 


No  tendency  for  pilot  to  induce  undesirable  motions 

Undesirable  motions  tend  to  occur  when  pilot  initiates  abrupt 
maneuvers  or  attempts  tight  control.  These  motions  can  be 
prevented  or  eliminated  by  pilot  technique 


Undesirable  motions  easily  induced  when  pilot  initiates  abrupt 
maneuvers  or  attempts  tight  control.  These  motions  can  be  g 

prevented  or  eliminated  but  only  at  sacrifice  to  task  performance 
or  through  considerable  pilot  attention  and  effort 

Ocillations  tend  to  develop  when  pilot  initiates  abrupt  maneuvers 
or  attempts  tight  control.  Pilot  must  reduce  gain  or  abandon  task  4 
to  recover 


Divergent? 


Pilot  Initiates 
Abrupt  Maneuvers 
or  Tight  Control 


Divergent  oscillations  tend  to  develop  when  pilot  initiates  abrupt 
maneuvers  or  attempts  tight  control.  Pilot  must  open  loop  by 
releasing  or  freezing  slick 


Causes 

Divergent 

Oscillation? 


(Disturbance  or  normal  pilot  control  may  cause  divergent  oscillation. 
Pilot  must  open  control  loop  by  releasing  or  freezing  the  stick 


Pilot  Attempts 
to  Enter  Control 
Loop 
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Appendix  B:  Simulink  Diagrams 


B.l  Top  Tier  View 


B.1.1  UA  Initial  Conditions 
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B.1.2  Unmanned  Aircraft  System 


B.l.2.1  Autopilot  Modem  from  Ground  Station 


Chi  dot 


omc 


From 


-KZ3 

Chi  dot  cmd 


V  dot  omd 


Froml 


-KZD 

V  dot  cmd 
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B.l.2.2  UA  Model 


To  Woilcspaoel 


B.l. 2.2.1  X1-X4  Integration 
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B.  1 .2.2.2  Embedded  MATLAB  Function 


function  [h, mu, x_dot ] =AC_EOMs (u, x) 

%  EOMs  for  a  PT  MASS  A/C  at  Constant  Altitude 

%  ***  QUANTITY  ******  UNITS  ********************************************* 
%  ***  length  ->  {ft} 

%  ***  velocity  ->  {ft/s} 

%  ***  accelerations  {ft/sA2} 

%  ***  angles  ->  {radians}  (calculations) 

%  ***  ang.  vel.  ->  {rad/s} 

kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk 

%  EXTRACT  INPUTS  FORM  INPUT  VECTOR 

%%  Inputs  used  for  constant  altitude  point-mass  simulation 
chi_dot_cmd  =  u(l);  %  Turn  Rate  Command  (radians/sec) 

V_dot_cmd  =  u(2);  %  Acceleration  Command  {ft/sA2} 
h_cmd  =  u(3);  %  Commanded  Altitude  {ft} 


EXTRACT  STATES  FROM  INPUT  STATE  VECTOR 

V  =  x(l);  %  Velocity  {ft/s} 

chi  =  x( 2);  %  Heading  {radians} 

N_pos  =  x( 3);  %  North  Position  {ft} 

E_pos  =  x(4);  %  East  Postion  {ft} 


%  Define  and/or  Calculate  Necassary  Constants 
d2r=pi/180; 
r2d=180/pi; 
g=32 . 17  ; 


%  NONLINEAR  Point  Mass  EQUATION  OF  MOTION  (EOMs) 

%  %  These  are  the  state  derivative  equations;  the  comment  names  the  state 
%  %  but  the  equation  is  for  its  derivative  (rate) 


%  Velocity 
V  dot=V  dot  cmd; 


%  Heading 

chi_dot=chi_dot_cmd; 

%  North  Position 
N_dot=V*cos (chi) ; 

%  East  Position 
E_dot=V*sin (chi) ; 

%  Pack  derivatives  into  output  vector  x_dot 
x_dot= [ 0 ; 0 ; 0 ; 0 ] ; 
x_dot(l)  =  V_dot; 
x_dot(2)  =  chi_dot; 
x_dot(3)  =  N_dot; 
x_dot(4)  =  E_dot; 

%  Finally,  calculate  bank  angle,  which  can  be  estimated  based  on  known 
%  states  and  rates 
h=0  ; 
mu=0; 
h = h  cmd; 

mu=atan2 (V*chi_dot, g) ; 
return 
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B.l.2.3  Camera  Model 
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CD - 

my  AC  Emulation  States 


B.  1 .2.3.1  Camera  Model  Transformation  into  AVDS  Form 


<\_po&> 
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phi_r2avds 
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pitch  theta_r2avds 
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[Hh-* 


■WD5- 

fps2kts 


0-Mi — H 


%%%  AVDS  Inputs  %%% 

1.  N{ft} 

2-  E  {ft} 

3.  h  {ft} 

4.  phi  (rol)  {AVDS  "units"  =  rad*2deg} 

5  theta  (pitch)  {AVDS  "units"  =  rad*2/deg} 

6  psi  (yaw)  {AVDS  "units"  =  -  radA2  deg} 

7.  AoA  {rad} 

3.  Seta  {rad} 

9.  nz  {normal  aocel)  {g's} 

10.  V  {kts} 

11  Ma 
12.-25. 


-*>03 


AVDS  AC  Simulation  Stock  States 


<\_po&> 


<E_pOS> 


<pri_aucs> 


<p6i_acs> 


To  AVDS 
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B.l.2.4  COT  Control  System  Embedded  MATLAB  Function 


function  [Chi  dot  cmd,V  dot  cmd]  =  CoT (Mouse  Xpos, Mouse  Ypos,mu,V) 
%  COT  Control  System 


%  DEFINE  Camera  Geometry 

CHFOV=60*pi/180;  %  Camera  Horizontal  Field  of  View 

%  For  future  use 


CVFOV=60*pi/180;  % 

Ctheta=-45*pi/180;  % 

set  in  AVDS) 
Cpsi=0*pi/180;  % 

set  in  AVDS) 

Czoom=l;  % 

%  Set  Constants 

mu  max=45*pi/180; 


Camera  Vertical  Field  of  View 

Camera  "pitch"  angle  above  a/c  nose  (currently 
Camera  "yaw"  angle  right  of  a/c  nose  (currently 
Camear  zoom  (currently  set  in  AVDS) 


g=32 . 17; 

V_dot_max=l*g;  %{ft/sA2} 

V_dot_min=-l*g;  %{ft/sA2} 

%  delta  chi=-Mouse  Xpos*CHF0V/2 ; 

Kpp= . 1 ; 

%  Chi  dot  cmd=Kpp*delta  chi; 

%  Chi_dot_cmd=Kpp*tan (delta_chi ) ; 

%  Chi  dot  cmd= . 1 *tan ( -Mouse  Xpos*pi/2); 

Kvdot=125; 

%V_dot  cmd=Kvdot*tan (Mouse  Ypos*pi/2); 

%mu=0;  %  Wings  level  demo 

delta  H=Mouse  Xpos*cos (mu) -Mouse  Ypos*sin (mu) ; 
delta  V=Mouse  Xpos*sin (mu) +Mouse  Ypos*cos (mu) ; 

%V  dot  cmd=Kvdot*Mouse  Ypos; 

V  dot  cmd=Kvdot*delta  V; 

%Chi  dot  cmd= . l*tan (-delta  E*pi/2); 
%V_dot_cmd=Kvdot*tan (delta_N*pi/2 ) ; 
delta  chi=-delta  H*(CHF0V/2); 

Chi  dot  cmd=Kpp* tan (delta  chi)  ; 

%  Set  Turn-Rate  Limit  based  on  bank  angle 
chi  dot  max=g/V*tan (mu  max)  ; 

%Now  check  if  commanded  turn  rate  inside  limits 
if (Chi  dot  cmd  >  chi  dot  max) 

Chi  dot  cmd=chi  dot  max; 
elseif (Chi  dot  cmd  <  -chi  dot  max) 

Chi  dot  cmd=-chi  dot  max; 


end 

%  Set  Acceleration  Limit  based  on  UA  performance  limits 
%Now  check  if  commanded  accel  inside  limits 
if (V  dot  cmd  >  V  dot  max) 

V  dot  cmd=V  dot  max; 
elseif (V  dot  cmd  <  V  dot  min) 

V  dot  cmd=V  dot  min; 

End 
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B.l.2.5  Ground  Station  Modem  to  Autopilot 


Chi_dot_cmd 


id^>< 


Chi  dot  cmdl 


<13 


-CZ3 


B.1.3  Target  Model 
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Integrator 


Target  Altitude 
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E_dot_tgt_fps 

ch  i_dot_tgt_cmd_dps 

h_tgt_ft 
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4€000 
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:_l  - 


T arget  East o  {ft} 


Integrator^ 


- ► 


Embedded 
MAT  LAB  Function  1 


Target  Surface 
Deflections 


EH 


Network  Send  Connection 


%  AVDS  Network  Blodc  Inputs 
1.  North  Position  {ft  from  initial  position} 
2  East  Position  {ft  from  initial  position} 
3.  Altitude  {ft} 

4.  Bank  Angle  {deg} 

5.  Pitch  Angle  {deg} 

6  Heading  (Yaw)  Angle  {deg} 
7.-10.  Surfaoe  Deflections 


B .  1 . 3 . 1  T arget  Embedded  MATLAB  Function 


function 

[N_dot_tgt_fps , E_dot_tgt_fps , h_tgt_ft, mu_tgt_deg, theta_tgt_deg, chi^tgt 
deg]  = 

target (V  tgt  cmd  kt,chi  dot  tgt  cmd  dps, chi  tgt  cmd  deg,h  tgt  cmd  ft) 

d2r=pi/180;  %  degrees  to  radians 
r2d=180/pi;  %  radians  to  degrees 
kt2fps=l . 689;  %knots  to  feet  per  second 
g_fpss=32.2;  %  accel  due  to  gravity  {fpsA2} 

V  tgt  fps=V  tgt  cmd  kt*kt2fps; 

N_dot_tgt_fps=V_tgt_fps*cos (chi_tgt_cmd_deg*d2r) ; 

E  dot  tgt  fps=V  tgt  fps*sin(chi  tgt_cmd  deg*d2r) ; 
h  tgt  ft=h  tgt  cmd  ft; 

mu  tgt  deg=atan2 (V  tgt  fps*chi  dot  tgt  cmd  dps*d2r,g  fpss) *r2d; 

theta_tgt_deg=0 ; 

chi  tgt  deg=chi  tgt  cmd  deg; 
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14.  ABSTRACT 

The  research  presented  in  this  thesis  focuses  on  developing,  demonstrating,  and  evaluating  the  concept  of  a  Cursor-on-Target  control  system  for 
semi-autonomous  unmanned  aircraft  systems.  The  Department  of  Defense  has  mapped  out  a  strategy  in  which  unmanned  aircraft  systems  will  increasingly 
replace  piloted  aircraft.  During  most  phases  of  flight  autonomous  unmanned  aircraft  control  reduces  operator  workload,  however,  real-time  information 
exchange  often  requires  an  operator  to  relay  decision  changes  to  the  unmanned  aircraft.  The  goal  of  this  research  is  to  develop  a  preliminary  Cursor-on-Target 
control  system  to  enable  the  operator  to  guide  the  unmanned  aircraft  with  minimal  workload  during  high  task  phases  of  flight  and  then  evaluate  the  operator’s 
ability  to  conduct  the  mission  using  that  control  system.  For  this  research,  the  problem  of  Cursor-on-Target  control  design  has  multiple  components.  Initially, 
a  Cursor-on-Target  controller  is  developed  in  Simulink.  Then,  this  controller  is  integrated  into  the  Aviator  Visual  Design  Simulator  to  develop  an  operator-in- 
the-loop  test  platform.  Finally,  a  ground  target  is  simulated  and  tracked  to  validate  the  Cursor-on-Target  controller.  The  Cursor-on-Target  control  system  is 
then  evaluated  using  a  proposed  operator  rating  scale. 
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